Image Processing Apparatus And Focus Adjusting Method

ABSTRACT

Provided is an image processing apparatus which is capable of easily selecting an optimum focus position for an inspection object even when a plurality of candidate focus positions are present. The image processing apparatus according to the invention includes: an imaging unit for imaging a region including an inspection object; a display unit for displaying an image; a focus adjusting unit for adjusting a focus position with respect to the inspection object; and an image processing unit for executing image processing on image data. A plurality of positions where part of the imaged region comes into a focused state are extracted as candidate focus positions, while the focus position with respect to the inspection object is changed, and candidate focus position information is displayed in the display unit. Selection of a focus position is accepted among the plurality of displayed candidate focus positions.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims foreign priority based on Japanese PatentApplication No. 2011-256012, filed Nov. 24, 2011, the contents of whichis incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, a focusadjusting method, and a computer program, which are capable ofextracting a plurality of candidate focus positions and selecting anoptimum focus position among the plurality of extracted candidate focuspositions while viewing those positions.

2. Description of Related Art

In the case of inspecting whether or not a defect is present in aninspection object, an appearance of the inspection object is imaged byan imaging device to determine whether or not the defect is presentbased on the imaged image. Proper determination of the presence of thedefect requires the imaged image to be clear, and normally, theinspection object is imaged after focus adjustment is performed. Manualadjustment or automatic adjustment is performed as the focus adjustment,where the automatic adjustment is easier to use for the user.

In the case of automatically performing the focus adjustment, anadjustment region such as a rectangular region or a circular region foradjusting a focus is set, and the focus degree of each of imaged imageswith regard to an inspection object imaged within the set adjustmentregion is evaluated, to extract a position with the highest focus degreeas a focus position. For example, Japanese Unexamined Patent PublicationNo. 2007-316432 discloses an enlarging observation device forcalculating focus values each indicating the degree of focus by animaging unit to extract a maximum focus value which is a focus valuebeing maximal, and moving the imaging unit to a maximum focus positioncorresponding to the maximum focus value, to thereby adjust a subjectdistance from the imaging unit to the inspection object.

The focus degree has hitherto been evaluated based on a quantity ofhigh-frequency components of image data. That is, it has been evaluatedthat a position with a larger quantity of high-frequency componentsbeing within the set adjustment region is more focused, and a positionwith a smaller quantity of higher-frequency component is less focused.

However, in the case of extracting the high-frequency component onlywithin the set adjustment region, there may occur a case where anunintended evaluation is made due to an influence of a size of theadjustment region or an influence of blurring on the periphery of theadjustment region. FIGS. 1A and 1B are schematic views for describing aproblem of the conventional evaluation of the focus degree.

As shown in FIG. 1A, in a so-called non-focused state in which the imageis not focused, a blurred portion 501 is present on the periphery of acharacteristic portion 500 of the inspection object, as compared with aso-called focused state in which the image is focused, shown in FIG. 1B.When an adjustment region 502 is set in this state, the background coloris one in the focused state of FIG. 1B, whereas a color different fromthe background color is mixed due to the presence of the blurred portion501 in the non-focused state of FIG. 1A. As a result, the quantity ofhigh-frequency components is likely to be determined to be larger thanthat in FIG. 1B, and hence the focus degree might be wrongly recognizedas higher in the non-focused state shown in FIG. 1A.

Further, in the non-focused state shown in FIG. 1A, since the imagebecomes an image including the blurred portion 501 being present at theperiphery of the characteristic portion 500 of the inspection object,setting of an appropriate adjustment region 502 itself is difficult.That is, in the case of the blurred portion 501 being present, it maynot be possible to properly perform the focus adjustment depending onsetting of the window (adjustment region). Moreover, “requiring thefocus adjustment” means the image is not focused, and since the user ismade to perform an operation for setting the window (adjustment region)on the non-focused image, setting of the window (adjustment region) hasbecome more difficult.

SUMMARY OF THE INVENTION

The present invention has been made in view of the above problems, andan object thereof is to provide an image processing apparatus, a focusadjusting method, and a computer program, which are capable of easilyselecting an optimum focus position for an inspection object even when aplurality of candidate focus positions are present.

In order to achieve the above object, according to one embodiment of theinvention, an image processing apparatus includes: an imaging unit (aimaging portion) for imaging an imaged region including an inspectionobject; a display unit (a display portion) for displaying an imageimaged by the imaging unit; a focus adjusting unit (a focus adjustingportion) having a focus adjustment mechanism for adjusting a focusposition with respect to the inspection object; and an image processingunit (an image processing portion) for executing image processing onimage data captured (acquired) by imaging with the imaging unit, whereinthe focus adjusting unit includes a candidate focus position extractingportion for extracting as candidate focus positions a plurality ofpositions where part of the imaged region comes into a focused statebased on a plurality of pieces of different image data captured byimaging the inspection object by the imaging unit, while changing thefocus position with respect to the inspection object by the focusadjustment mechanism, and a candidate displaying portion for displayingcandidate focus position information concerning the plurality ofextracted candidate focus positions in the display unit.

Further, according to another embodiment of the invention, in the imageprocessing apparatus according to the first aspect, the candidatedisplaying portion displays in the display unit the candidate focusposition information concerning the plurality of extracted candidatefocus positions so as to be selectable by the user, the focus adjustingunit is provided with a selection accepting portion for acceptingselection of one candidate focus position among the plurality ofcandidate focus positions displayed in the display unit, and when theselection of one candidate focus position is accepted by the selectionaccepting portion, the focus adjusting unit displays in the display unitan image in which the focus position is adjusted to the at least onecandidate focus position, the selection of which has been accepted.

Further, according to still another embodiment of the invention, in theimage processing apparatus according to the first or second aspect, thefocus adjusting unit is provided with a selection accepting portion foraccepting selection of one candidate focus position among the pluralityof extracted candidate focus positions, and when the selection of onecandidate focus position is accepted by the selection accepting portion,the focus adjusting unit adjusts the focus position to the one candidatefocus position, the selection of which has been accepted.

Further, according to still another embodiment of the invention, in theimage processing apparatus according to the first or second aspect, thefocus adjusting unit is provided with a selection accepting portion foraccepting selection of one candidate focus position, and when theselection of one candidate focus position is accepted by the selectionaccepting portion, the candidate displaying portion displays in thedisplay unit an image imaged upon adjusting the focus position to theone candidate focus position, the selection of which has been accepted.

Further, according to still another embodiment of the invention, in theimage processing apparatus according to the first or second aspect, thecandidate displaying portion displays in the display unit a candidatefocus position having a shortest distance from the imaging unit amongthe plurality of candidate focus positions.

Further, according to still another embodiment of the invention, in theimage processing apparatus according to any one of the first to fifthaspects, the focus adjusting unit is provided with a focus degreecalculation portion for calculating a focus degree for indicating adegree of a focused state with regard to each of a plurality ofpreviously set small regions, and a focus position acquiring portion foracquiring a focus position with a maximum focus degree among the focuseddegrees each calculated for each of the small regions, and the candidatefocus position extracting portion extracts the candidate focus positionsbased on the acquired focus position for each of the small regions.

Next, in order to achieve the above object, according to still anotherembodiment of the invention, there is provided a focus adjusting methodexecutable by an image processing apparatus, the apparatus including: animaging unit for imaging an imaged region including an inspectionobject; a display unit for displaying an image imaged by the imagingunit; a focus adjusting unit having a focus adjustment mechanism foradjusting a focus position with respect to the inspection object; and animage processing unit for executing image processing on image datacaptured by imaging with the imaging unit, wherein the focus adjustingunit extracts as candidate focus positions a plurality of positionswhere part of the imaged region comes into a focused state based on aplurality of pieces of different image data captured by imaging theinspection object by the imaging unit, while changing the focus positionwith respect to the inspection object by the focus adjustment mechanism,and displays candidate focus position information concerning theplurality of extracted candidate focus positions in the display unit.

Further, according to still another embodiment of the invention, in thefocus adjusting method according to the seventh aspect, the candidatefocus position information concerning the plurality of extractedcandidate focus positions are displayed in the display unit so as to beselectable by the user in the display unit, the focus adjusting unitaccepts selection of one candidate focus position among the plurality ofcandidate focus positions displayed in the display unit, and when theselection of one candidate focus position is accepted, the focusadjusting unit displays in the display unit an image in which the focusposition is adjusted to at least one candidate focus position, theselection of which has been accepted.

Further, according to still another embodiment of the invention, in thefocus adjusting method according to the seventh or eighth aspect, thefocus adjusting unit accepts selection of one candidate focus positionamong the plurality of extracted candidate focus positions, and when theselection of one candidate focus position is accepted, the focusadjusting unit adjusts the focus position to the one candidate focusposition, the selection of which has been accepted.

Further, according to still another embodiment of the invention, in thefocus adjusting method according to the seventh or eighth aspect, thefocus adjusting unit accepts selection of one candidate focus position,and when the selection of one candidate focus position is accepted, thefocus adjusting unit displays in the display unit an image imaged uponadjusting the focus position to the one candidate focus position, theselection of which has been accepted.

Further, according to still another embodiment of the invention, in thefocus adjusting method according to the seventh or eighth aspect, thecandidate displaying portion displays in the display unit a candidatefocus position having a shortest distance from the imaging unit amongthe plurality of candidate focus positions.

Further, according to still another embodiment of the invention, in thefocus adjusting method according to any one of the seventh to eleventhaspects, the focus adjusting unit calculates the focus degree forindicating the degree of a focused state with regard to each of aplurality of previously set small regions, acquires a focus positionwith a maximum focus degree among the focused degrees each calculatedfor each of the small regions, and extracts the candidate focuspositions based on the acquired focus position for each of the smallregions.

Next, in order to achieve the above object, according to still anotherembodiment of the invention, there is provided a computer programexecutable by an image processing apparatus, the apparatus including: animaging unit for imaging an imaged region including an inspectionobject; a display unit for displaying an image imaged by the imagingunit; a focus adjusting unit having a focus adjustment mechanism foradjusting a focus position with respect to the inspection object; and animage processing unit for executing image processing on image datacaptured by imaging with the imaging unit, wherein the focus adjustingunit is caused to function as a candidate focus position extractingportion for extracting as candidate focus positions a plurality ofpositions where part of the imaged region comes into a focused statebased on a plurality of pieces of different image data captured byimaging the inspection object by the imaging unit, while changing thefocus position with respect to the inspection object by the focusadjustment mechanism, and a candidate displaying portion for displayingcandidate focus position information concerning the plurality ofextracted candidate focus positions in the display unit.

Further, according to still another embodiment of the invention, in thecomputer program according to the thirteenth aspect, the candidatedisplaying portion is caused to function as a unit for displaying in thedisplay unit the candidate focus position information concerning theplurality of extracted candidate focus positions so as to be selectableby the user in the display unit, the focus adjusting unit is caused tofunction as a selection accepting portion for accepting selection of onecandidate focus position among the plurality of candidate focuspositions displayed in the display unit, and when the selection of onecandidate focus position is accepted, the focus adjusting unit is causedto function as a unit for displaying in the display unit an image inwhich the focus position is adjusted to the at least one candidate focusposition, the selection of which has been accepted.

Further, according to still another embodiment of the invention, in thecomputer program according to the thirteenth or fourteenth aspect, thefocus adjusting unit is caused to function as a selection acceptingportion for accepting selection of one candidate focus position amongthe plurality of extracted candidate focus positions, and when theselection of one candidate focus position is accepted, the focusadjusting unit is caused to function as a unit for adjusting the focusposition to the one candidate focus position, the selection of which hasbeen accepted.

Further, according to still another embodiment of the invention, in thecomputer program according to the thirteenth or fourteenth aspect, thefocus adjusting unit is caused to function as a selection acceptingportion for accepting selection of one candidate focus position, andwhen the selection of one candidate focus position is accepted, thecandidate displaying portion is caused to function as a unit fordisplaying in the display unit an image imaged upon adjusting the focusposition to the one candidate focus position, the selection of which hasbeen accepted.

Further, according to still another embodiment of the invention, in thecomputer program according to the thirteenth or fourteenth aspect, thecandidate displaying portion is caused to function as a unit fordisplaying in the display unit a candidate focus position having ashortest distance from the imaging unit among the plurality of candidatefocus positions.

Further, according to still another embodiment of the invention, in thecomputer program according to any one of the thirteenth to seventeenthaspects, the focus adjusting unit is caused to function as a focusdegree calculation portion for calculating the focus degree forindicating the degree of a focused state with regard to each of aplurality of previously set small regions, and a focus positionacquiring portion for acquiring a focus position with a maximum focusdegree among the focused degrees each calculated for each of the smallregions, and the candidate focus position extracting portion is causedto function as a unit for extracting the candidate focus positions basedon the acquired focus position for each of the small regions.

In the first, seventh, and thirteenth aspects, a plurality of positionswhere part of the imaged region comes into a focused state are extractedas candidate focus positions based on a plurality of pieces of differentimage data captured by imaging the inspection object by the imagingunit, while the focus position with respect to the inspection object ischanged by the focus adjustment mechanism. Candidate focus positioninformation concerning the plurality of extracted candidate focuspositions is displayed in the display unit. A plurality of candidatefocus positions are extracted and focus adjustment can be performedwhile checking on a screen as to which part of the imaged region is in afocused state. Accordingly, it is possible for the user to easily selecta focus position corresponding to an image in which a desiredcharacteristic portion of the inspection object is clear.

In the second, eighth, and fourteenth aspects, the candidate focusposition information concerning the plurality of extracted candidatefocus positions are displayed in the display unit so as to be selectableby the user in the display unit, selection of one candidate focusposition is accepted among the plurality of candidate focus positionsdisplayed in the display unit, and when the selection of one candidatefocus position is accepted, an image in which the focus position isadjusted to the at least one candidate focus position, the selection ofwhich has been accepted, is displayed in the display unit. Accordingly,it is possible to display an image corresponding to the candidate focusposition, so as to visually check whether or not the selected imagedregion is proper.

In the third, ninth, and fifteenth aspects, selection of one candidatefocus position is accepted among the plurality of extracted candidatefocus positions, and when the selection of one candidate focus positionis accepted, the focus position is adjusted to the one candidate focusposition, the selection of which has been accepted. Accordingly, it ispossible to display an image corresponding to the candidate focusposition, so as to visually check whether or not the selected imagedregion is proper.

In the fourth, tenth, and sixteenth aspects, an image imaged uponadjusting the focus position to one candidate focus position, theselection of which has been accepted, is displayed in the display unit.Accordingly, it is possible to visually check whether or not a desiredcharacteristic portion of the inspection object is clear, so as toeasily select a focus position corresponding to a desired imaged region.

In the fifth, eleventh, and seventeenth aspects, a candidate focusposition having a shortest distance from the imaging unit among theplurality of candidate focus positions is displayed in the display unit.Since a candidate focus position, which has a shortest distance from theimaging unit and most likely to come into the focused state, isdisplayed, there is a high probability that the focus adjustment becomesunnecessary, and a complicated operation can be omitted in some cases.

In the sixth, twelfth, and eighteenth aspects, the focus degree forindicating the degree of a focused state with regard to each of aplurality of previously set small regions is calculated, to acquire afocus position with a maximum focus degree among the focused degreeseach calculated for each of the small regions. Since the candidate focuspositions are extracted based on the acquired focus position for each ofthe small regions, it is possible to extract a candidate focus positionlikely to come into the focused state.

In the present invention, a plurality of candidate focus positions areextracted and focus adjustment can be performed while checking on ascreen as to which part of the imaged region is in a focused state.Accordingly, it is possible for the user to easily select a focusposition corresponding to an image in which a desired characteristicportion of the inspection object is clear.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are schematic views for describing a problem of aconventional evaluation of a focus degree.

FIG. 2 is a schematic view showing a configuration of an imageprocessing sensor according to an embodiment of the present invention;

FIGS. 3A to 3C are outline views showing a configuration of an imagingdevice of the image processing sensor according to the embodiment of thepresent invention;

FIG. 4 is a block diagram showing a hardware configuration of theimaging device of the image processing sensor according to theembodiment of the present invention;

FIG. 5 is a perspective view showing a configuration of a focusadjustment mechanism of a camera module in the imaging device of theimage processing sensor according to the embodiment of the presentinvention;

FIG. 6 is a front view showing a configuration of a display device ofthe image processing sensor according to the embodiment of the presentinvention;

FIGS. 7A and 7B are exemplary views of mode switching screens in thedisplay device of the image processing sensor according to theembodiment of the present invention;

FIGS. 8A to 8F are exemplary views of setting screens in the displaydevice of the image processing sensor according to the embodiment of thepresent invention;

FIG. 9 is a function block diagram of the image processing sensoraccording to the embodiment of the present invention;

FIGS. 10A and 10B are schematic views for describing a focus degreecalculating method of the image processing sensor according to theembodiment of the present invention;

FIGS. 11A to 11C are schematic views for describing a focus degreecalculating method based on stored image data, performed by the imageprocessing sensor according to the embodiment of the present invention;

FIG. 12 is a table for describing a candidate focus position extractingmethod based on a calculated focus degree, performed by the imageprocessing sensor according to the embodiment of the present invention;

FIG. 13 is a table for describing a candidate focus position extractingmethod in a case where the calculated focus degree varies, the methodbeing performed by the image processing sensor according to theembodiment of the present invention; and

FIGS. 14A to 14G are exemplary views of a high-frequency extractedimage, created by the image processing sensor according to theembodiment of the present invention;

FIG. 15 is an exemplary view showing arrangement of the focus degrees,representing for each pixel an image number of the high-frequencyextracted image with a maximum focus degree;

FIGS. 16A to 16C are exemplary views of an automatic focus adjustmentscreen in the display device of the image processing sensor according tothe embodiment of the present invention;

FIG. 17 is a flowchart showing a procedure for automatic focusadjustment processing of an FPGA and a DSP of a main substrate in theimaging device of the image processing sensor according to theembodiment of the present invention;

FIGS. 18A to 18C are exemplary views of a candidate-focus positionselection accepting screen of the display device in the case ofdisplaying candidate focus position information by selection of acandidate button on a touch panel;

FIGS. 19A to 19C are exemplary views of the candidate-focus positionselection accepting screen of the display device in the case ofaccepting the selection of the candidate focus position by specificationof a target focus position on the touch panel;

FIGS. 20A and 20B are exemplary views of a focused-state selectionaccepting screen of the display device where the focused state is madesequentially selectable;

FIGS. 21A to 21D are exemplary views of the candidate-focus positionselection accepting screen of the display device in the case of settinga small region for each candidate focus position; and

FIG. 22 is a flowchart showing a procedure for automatic focusadjustment processing based on a high-frequency extracted image,performed by a DSP of a main substrate in the imaging device of theimage processing sensor according to the embodiment of the presentinvention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, an image processing apparatus according to an embodiment ofthe present invention will be described with reference to the drawings.It is to be noted that elements having the same or similarconfigurations or functions throughout the drawings referenced indescriptions of the present embodiment are provided with the same orsimilar reference numerals, and detailed descriptions thereof areomitted. Hereinafter, descriptions will be made, giving a case of aprocessing sensor as an image processing apparatus.

FIG. 2 is a schematic view showing a configuration of an imageprocessing sensor according to an embodiment of the present invention.As shown in FIG. 2, the image processing sensor according to the presentembodiment is configured by an imaging device 1 and a display device 2connected with the imaging device 1 through a connection cable 3 in adata communicable manner.

Needless to say, the image processing sensor may be an external computerhaving a display in place of the display device 2. The imaging device 1and the display device 2 may be integrally formed.

The imaging device 1 includes therein an FPGA, DSP, and the like whichexecute image processing, and provided with a camera module (imagingunit) having an imaging element for imaging an inspection object, and anillumination part for irradiating the inspection object with light. Inorder to make the imaging device 1 compact, for example, as shown inFIG. 1, a lens 12 is arranged close to the center of the front face ofthe imaging device 1, and a plurality of LEDs 11 are arranged as theillumination part so as to surround the periphery of the lens 12. It isto be noted that external illumination (ring illumination, or the like)may be provided separately from the imaging device 1.

FIGS. 3A to 3C are outline views showing a configuration of the imagingdevice 1 of the image processing sensor according to the embodiment ofthe present invention. FIG. 3A is a front view showing the configurationof the imaging device 1 of the image processing sensor according to theembodiment of the present invention, FIG. 3B is a plan view showing theconfiguration of the imaging device 1 of the image processing sensoraccording to the embodiment of the present invention, and FIG. 3C is arear view showing the configuration of the imaging device 1 of the imageprocessing sensor according to the embodiment of the present.

As shown in FIG. 3A, the lens 12 is arranged close to the center of thefront face of the imaging device 1, and the plurality of LEDs 11 arearranged so as to surround the periphery of the lens 12. At the time ofimaging, the plurality of LEDs 11 are turned on, to irradiate an imagedregion including the inspection object with light, thereby allowingclear imaging of the imaged region including the inspection object.

As shown in FIGS. 3B and 3C, the imaging device 1 includes, on its rearface, a power source connector 102 to be connected with a power cablethat receives supply of electric power from an external power source,and a connection connector 103 connectable with the connection cable 3that performs data communication with the display device 2. Further, theimaging device 1 also includes, on the rear face, a focus adjustingscrew 101 capable of manually adjusting a focus.

FIG. 4 is a block diagram showing a hardware configuration of theimaging device 1 of the image processing sensor according to theembodiment of the present invention. In FIG. 4, a connector substrate 16receives supply of electric power from the external power source via thepower source connector 102 (see FIGS. 3B and 3C) provided in a powerinterface 161. A power substrate 18 supplies the supplied electric powerto each substrate. In the present embodiment, electric power is suppliedto a camera module 14 via a main substrate 13.

A motor driver 181 of the power substrate 18 supplies drive electricpower to a motor 141 of the camera module 14, to realize auto focusing.That is, the motor driver 181 and the motor 141 function as an exampleof the focus adjustment mechanism.

A communication substrate 17 transmits to the display device 2 an OK/NGsignal (determination signal) outputted from the main substrate 13 andindicating failure/non-failure of the inspection object in accordancewith whether or not a defect has been detected. The display device 2having received the determination signal displays a result of thedetermination.

An illumination substrate (illumination part) 15 irradiates the imagedregion for imaging the inspection object. An illumination substrate 15is provided with the plurality of LEDs 11, and is provided with areflector (not shown), on the front of the plurality of LEDs 11.Further, the lens 12 is interchangeable as a lens unit for a shortdistance or a long distance.

The camera module (imaging unit) 14 is provided with a focus adjustmentpart for controlling the auto focus motion by rotation of the motor 141,and after focus adjustment, the camera module 14 images the inspectionobject in accordance with an imaging instruction signal from the mainsubstrate 13. In the present embodiment, a CMOS substrate 142 isprovided as an imaging element, and an imaged color image is convertedto an HDR image based on a conversion characteristic for expanding adynamic range on the CMOS substrate 142, and is outputted to the FPGA131 of the main substrate 13. That is, the CMOS substrate 142 isprovided with a function to generate and output the HDR image, and forexample, an HDRCMOS or the like corresponds to this. In the imagingdevice 1 of the image processing sensor, the switching part may beprovided for switching between whether or not to convert the image tothe HDR image with the CMOS substrate. When a workpiece resistant tooccurrence of halation and the like is carried in the case of imagingthe inspection object, the switching part can make switching so as notto generate the HDR image, and can further alleviate an arithmeticprocessing load.

FIG. 5 is a perspective view showing a configuration of the focusadjustment mechanism of the camera module 14 in the imaging device 1 ofthe image processing sensor according to the embodiment of the presentinvention. As shown in FIG. 5, a base mount 143 is mounted on the CMOSsubstrate 142 of the camera module 14. The base mount 143 is providedwith a vertical detection sensor (not illustrated) that detects a motionin a vertical direction (arrow direction) of a lens support 121 thatsupports the lens 12 in a lens holder 144, and a rotation detectionsensor (not illustrated) that detects a rotation of the motor 141 aspart of a mechanism to move the lens support 121.

The base mount 143 is mounted with the lens holder 144 configured by thelens support 121 and the mechanism to move the lens support 121. Asshown in FIG. 5 an external thread formed at a rotational shaft 141 a ofthe motor 141 is screwed with an internal thread formed at a nut 141 b.Hence, the nut 141 b linearly moves in the arrow direction in accordancewith the rotation of the motor 141. The nut 141 b is arranged so as topush up the lens support 121 from below, and the lens support 121 ispushed down to below by a biasing member (not shown). Accordingly, bycontrolling a reciprocating motion in the arrow direction of the nut 141b by rotation of the motor 141, the lens support 121 can be raised orlowered against the biasing force of the biasing member by means of thenut 141 b, whereby a distance between the lens 12 provided in the lenssupport 121 and the inspection object can be adjusted.

Returning to FIG. 4, the main substrate 13 controls a motion of eachsubstrate having been connected therewith. For example, with respect tothe illumination substrate 15, a control signal for controllingturning-on/off of the plurality of LEDs 11 is transmitted to an LEDdriver 151. The LED driver 151, for example, adjusts turning on/off, anamount of light, and the like, of the LEDs 11 in accordance with thecontrol signal from the FPGA 131. Further, a control signal forcontrolling an auto focusing operation is transmitted to the motor 141of the camera module 14 via a motor driver 181 of the power substrate18, and an imaging instruction signal or the like is transmitted to theCMOS substrate 142.

While performing illumination control and imaging control, the FPGA 131of the main substrate 13 executes image processing on the captured imagedata (image processing unit). Further, a DSP 132 of the main substrate13 executes edge detection processing, pattern search processing, andthe like on the image data. As a result of the pattern searchprocessing, an OK/NG signal (determination signal), indicatingfailure/non-failure of the inspection object in accordance with whetheror not a defect has been detected is outputted to the communicationsubstrate 17. A result of the arithmetic processing and the like arestored into a memory 133. In the present embodiment, processing with arelatively heavy load, such as the edge detection processing or thepattern search processing, is executed by the DSP 132. However, theprocessing may be executed by the FPGA 131 depending on the situation.In short, the image processing unit is sufficient as long as some kindof image processing can be executed on image data captured by the CMOSsubstrate 142.

Further, although the FPGA 131 performs illumination control, imagingcontrol, and the like in the present embodiment, the DSP 132 may performthe illumination control, the imaging control, and the like. Moreover, acircuit formed of the FPGA 131 united with the DSP 132, namely, a maincontrol circuit (main control unit), may be provided. In short, the maincontrol unit transmits the control signal for controlling turning-on/offof the plurality of LEDs 11 to the LED driver 151, transmits the controlsignal for controlling an auto focus motion to the motor 141 of thecamera module 14 via the motor driver 181 of the power substrate 18,transmits the imaging instruction signal or the like to the CMOSsubstrate 142, and has both functions of the FPGA 131 and the DSP 132.

FIG. 6 is a front view showing a configuration of the display device 2of the image processing sensor according to the embodiment of thepresent invention. As shown in FIG. 6, a touch panel 21 is provided atthe center portion of the front face of the display device 2, anddisplays a color image of an imaged inspection object on the screen,while accepting a selection input by the user.

Further, the display device 2 is provided with a power connector 24 tobe connected with the power cable supplied with electric power from theexternal power source, and a connection connector 25 connectable withthe connection cable 3 that performs data communication with the imagingdevice 1. Moreover, a USB port 22 connectable with a USB memory and thelike is provided on the front face.

The user selects a button displayed on the screen of the touch panel 21of the display device 2, to thereby control an operation of the imageprocessing sensor. It is also possible to switch between “inspectionmode” for executing an inspection of the inspection object and “settingmode” for performing a condition setting for the imaging device 1. Inother words, the image processing sensor according to the presentembodiment has a mode switching part for switching between theinspection mode (Run mode) for determining failure/non-failure of theinspection object and the setting mode (Non-Run mode) for setting avariety of parameters (imaging parameter, illumination parameter, imageprocessing parameter, and the like) which are used for the inspection.FIGS. 7A and 7B are exemplary views of mode switching screens in thedisplay device 2 of the image processing sensor according to theembodiment of the present invention.

FIG. 7A is an exemplary view of a screen display of “inspection mode”.As shown in FIG. 7A, an image of the inspection object imaged by theimaging device 1 is displayed in an inspection object displaying region71. A “SENSOR SETTING” button 72 at the lower left functions as theswitching part, and when the “SENSOR SETTING” button 72 is selected, themode is switched to “setting mode”, and the screen transitions to thescreen shown in FIG. 7B.

FIG. 7B is an exemplary view of a screen display of “setting mode”. Asshown in FIG. 7B, the kind of the inspection object is selected in aprogram selecting region 73. Herein, the “program” means a series ofdata groups (combination of parameter values) set in accordance with thekind of the inspection object or an inspection environment, and adifferent data group can be stored as the program for each kind of theinspection object. Inspection condition data concerning a condition forinspecting the inspection object includes focus position data indicatinga focus position in the case of performing focus adjustment.

Further, when a master image to become a reference for comparison withthe inspection object is stored, the master image is displayed in amaster image displaying region 74. When a “SETTING NAVIGATION” button 75is selected, the screen transitions to a setting screen for performingdetailed setting. A “START OPERATION” button 76 of FIG. 7B functions asa switching unit, and when the “START OPERATION” button 76 is selected,the mode is switched to the “inspection mode”, and the screentransitions to the screen shown in FIG. 7A.

FIGS. 8A to 8F are exemplary views of setting screens in the displaydevice 2 of the image processing sensor according to the embodiment ofthe present invention. Through the setting screens shown in FIGS. 8A to8F, the user sequentially performs setting in the flow of setting of theimaging condition (FIG. 8A), registration of the master image to becomea reference for pattern search (FIG. 8B), setting of a tool such as edgesearch on the master image (FIGS. 8C to 8E), and allocation of an output(FIG. 8F). Hereinafter, a detailed description will be given. When the“SETTING NAVIGATION” button 75 shown in FIG. 7B is selected, first, animaging condition setting screen shown in FIG. 8A is displayed. On theimaging condition setting screen, a currently imaged image of theinspection object is displayed when the master image is not stored, andthe master image is displayed in a main display region 81 when themaster image is stored, and a setting button group for setting imagingconditions is displayed in the lower part of the screen. For example,when a “TRIGGER CONDITION” button is selected, it is possible to set atrigger condition for specifying a timing when the imaging device 1images the inspection object. Although a detailed setting screen isomitted, when each button is selected, the setting screen is displayedon the touch panel 21 shown in FIG. 6 in accordance with each settingcondition. The case where an “AUTOMATIC FOCUS ADJUSTMENT” button 83 isselected will be described later.

Further, for more detailed setting, an “EXTENDED FUNCTION” button 82 ofFIG. 8A may be selected. When the “EXTENDED FUNCTION” button 82 isselected, a button for performing detailed setting is separatelydisplayed. As described above, on the imaging condition setting screen,it is possible to adjust brightness, adjust focus, and set imagingrange, on/off of illumination, on/off of zooming, and the like. Focusadjustment will be described later.

When a “screen transition” button 84 displayed as “NEXT” of FIG. 8A isselected, a master image registering screen shown in FIG. 8B isdisplayed. Hereinafter, a variety of tools for inspection will be set onthe registered master image. A plurality of programs can be stored withrespect to one master image. That is, different tools can be set withrespect to the same master image, and can be previously stored asdifferent programs.

As the master image, an image of the currently imaged inspection objectmay be registered, or an image selected from previously imaged imagesmay be registered. In the case of registering the currently imagedimage, the user may select the “REGISTER LIVE IMAGE” button 85. An imagebeing imaged at the time of selection of the “REGISTER LIVE IMAGE”button 85 is registered as the master image.

When a “screen transition” button 86 displayed as “NEXT” of FIG. 8B isselected, a tool setting screen shown for each master image, shown inFIG. 8C, is displayed. Hereinafter, a variety of tools for inspectionwill be set on the master image.

On the tool setting screen, a tool for executing the inspection isadditionally set in the displayed master image. When an “ADD” button 87shown in FIG. 8C is selected, a tool selecting screen shown in FIG. 8Dis displayed. A tool selected on the tool selecting screen isadditionally set. For example, when a “SEARCH EDGE” button 88 isselected, an edge search setting screen shown in FIG. 8E is displayed.By previously setting which edge of the master image is to be checkedwith the imaged image of the inspection object on the edge searchsetting screen, it is possible to determine failure/non-failure of theinspection object in accordance with whether or not a defect has beendetected. Hereinafter, a color area, positional correction, and the likecan be set.

When a “screen transition” button 89 displayed as “NEXT” of FIG. 8C isselected, an output allocating screen, shown in FIG. 8F, is displayed.When an “END” button 90 is selected, the screen display returns to“setting mode” shown in FIG. 7B. In this manner, by sequentiallyselecting the “screen transition” buttons 84, 86, 89 displayed as “NEXT”on the touch panel 21 of the display device 2 shown in FIG. 6, the usercan easily set in a short period of time a variety of parameters whichare used for the inspection. Further, since even a user who is not usedto the image processing sensor is guided to a next operation on thetouch panel 21 of the display device 2, a variety of parameters can beeasily set.

FIG. 9 is a function block diagram of the image processing sensoraccording to the embodiment of the present invention. In FIG. 9, animaging unit 901 images an imaged region including an inspection object.In the camera module 14 of FIG. 4, a function of the CMOS substrate 142corresponds to this.

A focus adjusting unit 902 adjusts a focus position with respect to theinspection object. The focus adjusting unit 902 is provided with acandidate focus position extracting portion (part) 911, a selectionaccepting portion (part) 912, a candidate displaying portion (part) 913,a focus degree calculating portion (part) 914, and a focus positionacquiring portion (part) 915.

The candidate focus position extracting portion (part) 911 extracts ascandidate focus positions a plurality of positions where part of theimaged region comes into a focused state based on a plurality of piecesof different image data captured by imaging the inspection object by thecamera module 14, while changing the focus position with respect to theinspection object by the focus adjustment mechanism. In order to extractthe candidate focus positions, the focus degree calculating portion(part) 914 calculates the focus degree for indicating the degree of thefocused state with regard to each of a plurality of small regionspreviously set as imaged regions, and the focus position acquiringportion (part) 915 acquires a focus position with a maximum focus degreeamong the focused degrees each calculated for each of the small regions.The candidate focus position extracting portion (part) 911 extracts theplurality of candidate focus positions based on the focus positionacquired by the focus position acquiring portion (part) 915 for each ofthe small regions.

FIGS. 10A and 10B are schematic views for describing a focus degreecalculating method of the image processing sensor according to theembodiment of the present invention. In FIG. 10A, a plurality of virtualcenters 1001 are set at twelve points on an imaged image. An adjustmentregion (small region) 1002 is set on the periphery of each of thevirtual centers 1001, being located several tens of pixels therefrom,and a high-frequency component is extracted in each of the adjustmentregions 1002. Specifically, as a ratio (percentage) of the number ofpixels of the high-frequency component to the number of pixels withinthe adjustment region 1002, the focus degree for each of the virtualcenters 1001 (each of the adjustments regions 1002) is previouslycalculated and stored.

Needless to say, the method is not restricted to setting of twelveadjustment regions 1002 as the small regions, and may be a method forexplicitly arranging a plurality of window regions. In FIG. 10B, aplurality of window regions 1003 are arranged on the imaged image. Ahigh-frequency component is extracted in each of the window regions1003, and as a ratio (percentage) of the number of pixels of thehigh-frequency component to the number of pixels within the windowregion 1003, the focus degree for each of the window regions 1003 ispreviously calculated and stored.

Although the focus degree calculating method shown in FIG. 10 is amethod for dynamically calculating the focus degree with respect to animage imaged by the imaging unit, a high-frequency component may befinely detected if a previously imaged and stored image is used. In thiscase, differently from FIG. 10, setting twelve virtual centers 1001 isnot necessary, and a detailed high-frequency extracted image can becreated on a full screen.

FIGS. 11A to 11C are schematic views for describing a focus degreecalculating method based on a stored image, performed by the imageprocessing sensor according to the embodiment of the present invention.A high-frequency component of an imaged image shown in FIG. 11A isextracted, to obtain a high-frequency extracted image shown in FIG. 11B.FIG. 11B shows that the high-frequency component is present in a whiteportion, and is not present in a black portion. Needless to say, ahigh-frequency extracted image may be created in the same size as thatof an original image, or the size of the high-frequency extracted imagemay be changed by reducing the size of the image or the like. Forexample, FIG. 11C shows a high-frequency extracted image of a reducedsize.

The focus position acquiring portion (part) 915 extracts a plurality ofcandidate focus positions based on a distribution of the focus degree orthe high-frequency extracted image. FIG. 12 is a table for describing acandidate focus position extracting method based on a calculated focusdegree, performed by the image processing sensor according to theembodiment of the present invention. It is to be noted that thecalculated focus degree may be weighed by being multiplied by a suitablefixed coefficient.

FIG. 12 shows the focus degree for each lens position at each virtualcenter point. Among virtual center points 1 to 12, the lens position(candidate focus positions) with a maximum focus degree at the virtualcenter points 1 to 8 is ‘20’, and the lens position with a maximum focusdegree at the virtual center points 9 to 12 is ‘35’. The lens positionis ‘0’ when it is in a contact state with the inspection object, and avalue of the lens position increases as the lens 12 moves away from theinspection object, that is, as the lens 12 is closer to the cameramodule 14.

In the example of FIG. 12, two lens positions ‘20’ and ‘35’ can beextracted as the candidate focus positions. The lens position ‘35’ iscloser to the camera module 14, that is, has a shorter focus distanceamong the plurality of candidate focus positions, and hence the lens 12is moved to the lens position ‘35’ at the time of automatic focusadjustment, to adjust the focus position to the candidate focus positionand complete the process.

As described with reference to FIGS. 10 to 12 in the present embodiment,the ratio (percentage) of the number of pixels of the high-frequencycomponent (or edge component) to the number of pixels within theadjustment region 1002 is calculated as the focus degree for each of thevirtual centers 1001(each of the adjustment regions 1002), but thecalculating method for the focus degree is not particularly restrictedthereto. For example, an l×m filter is vertically moved at every npixels with respect to an imaged image (l, m, n are all arbitraryintegers), and a variance value of pixel values of all pixels presentwithin an l×m region is calculated every time the filter is moved, tocreate a high-frequency image made up of two-dimensional array of thevariance value. With respect to the created high-frequency image,addition or multiplication for weighing the variance value by means ofpreviously decided weighing is performed, to calculate the focus degree(i.e., weighing performed by means of a coefficient based on acoordinate of the foregoing virtual center 1001, to calculate twelvefocus degrees at the respective virtual centers 1001). As describedabove, creating the high-frequency image also allows calculation of thefocus degree. In short, the calculation method for the focus degree isnot particularly limited as long as the focus degree shows the degree ofthe focused state for each of a plurality of previously set smallregions.

Herein, the focus degree actually varies more often than the case shownin the example of FIG. 12 where the focus degree does not vary. FIG. 13is a table for describing a candidate focus position extracting methodin a case where the calculated focus degree varies, the method beingperformed by the image processing sensor according to the embodiment ofthe present invention.

Differently from FIG. 12, in the example of FIG. 13, the focus degreevaries at the virtual center points 2, 6, 10. When such variationsoccur, the number of occurrence at each of the lens positions is sortedin the descending order, and the lens position is extracted as thecandidate focus position in the order from the lens position with thelargest number of occurrence. At the time of extraction, the focusdegree to be excluded from the candidate focus position is selected fromthe focus degree in the vicinity of the lens position as the extractedcandidate focus position.

For example in FIG. 13, since the number of occurrence of the lensposition ‘20’ is six times, which is the largest number, the lensposition ‘20’ is extracted as the candidate focus position. When a widthof the focus degree attributable to the same inspection object is ‘5’,the lens position ‘15’ to ‘25’ can be regarded as the same candidatefocus position, and the lens positions ‘15’ and ‘25’ can be excludedfrom the candidate focus positions.

Similarly, since the number of occurrence of the lens position ‘35’ isthree times, which is the second largest number, the lens position ‘35’is extracted as the candidate focus position. When the width of thefocus degree attributable to the same inspection object is ‘5’, the lensposition ‘30’ to ‘40’ can be regarded as the same candidate focusposition, and the lens positions ‘30’ and ‘40’ can be excluded from thecandidate focus positions. As described above, the lens positions ‘20’and ‘35’ are extracted as the candidate focus positions, and since thelens position ‘35’ has a shorter focus distance, automatic adjustment isperformed on the lens position ‘35’.

The focus position acquiring portion (part) 915 can also extractcandidate focus positions based on a stored high-frequency extractedimage created by extracting a high-frequency component of an imageobtained by imaging the inspection object. FIGS. 14A to 14G areexemplary views of a high-frequency extracted image, created by theimage processing sensor according to the embodiment of the presentinvention.

FIG. 14A to 14G respectively show original images on the left side andhigh-frequency extracted images based on the focus degrees on the rightside. A portion displayed in white in the high-frequency extracted imageon the right side shows a portion having a large quantity ofhigh-frequency component, that is, a portion having a focus degreelarger than a predetermined value.

The focus position acquiring portion (part) 915 selects a high-frequencyextracted image with a maximum focus degree for each pixel of thecreated high-frequency extracted image. FIG. 15 is an exemplary viewshowing arrangement of the focus degrees, representing for each pixel animage number of the high-frequency extracted image with a maximum focusdegree.

FIG. 15 represents the image number of the high-frequency extractedimage shown in FIG. 14 with the maximum focus degree for each pixel. Forexample, a pixel represented as ‘2’ indicates that the focus degree ofthe image number ‘2’ is maximal. A region where the same image numbersare sequentially represented can be specified as the focused region, andextracted as the candidate focus position.

It is to be noted that the value is often not stable on a boundary offocused regions, but in this case, the focused region itself, evenincluding the adjacent focused regions, does not significantlyfluctuate. Further, a pixel with a relatively small maximum value offocus degree is susceptible to noise. Therefore, a minimum value of thefocus degree that can be specified as the focused region may bepreviously set, and when the focus degree is not larger than the minimumvalue, the region may not be included in the focused region.

Further, when a blurred portion or the like is present in the originalimage, the region might be judged to have a large quantity ofhigh-frequency component, and wrongly recognized as being in the focusedstate. In order to avoid this, each pixel of the high-frequencyextracted image is not compared as it is, but a value subjected tofilter processing for each pixel with respect to the original image(image on the left side of FIG. 14), or a value obtained by calculatinga standard deviation and performing division thereby, may be compared toselect the high-frequency extracted image with a maximum value.

Moreover, the high-frequency component might be wrongly calculated in alarge quantity depending on a pattern. Thereat, for each pixel, aperipheral pixel value may be weighed to decide a pixel value, or amaximum value of the peripheral pixel value may be taken as arepresentative value to decide a pixel value, whereby a high-frequencyextracted image may be created based on the decided pixel value. In thismanner, an influence of an error due to the pattern can be reduced.Therefore, a region with a maximum focus degree for each pixel can behighly accurately specified as the focused region from thehigh-frequency extracted image, and can be extracted as the candidatefocus position.

Needless to say, by combination of the above methods, a region with amaximum focus degree for each pixel can be highly accurately specifiedas the focus region from the high-frequency extracted image, and can beextracted as the candidate focus position.

Returning to FIG. 9, the selection accepting portion (part) 912 acceptsselection of one candidate focus position among the plurality ofextracted candidate focus positions. The method for accepting theselection is not particularly limited, and any method can be used aslong as the selection of information from which the candidate focusposition displayed on the touch panel 21 of the display device 2 isidentifiable can be accepted.

When the selection of one candidate focus position is accepted with theselection accepting portion (part) 912, the focus position is adjustedto one candidate focus position, the selection of which has beenaccepted. Specifically, the DSP 132 (or this may be the FPGA 131)transmits a rotation instruction to the motor 141 of the camera module14 via the motor driver 181, to move the lens 12 and adjust the focusposition to the one candidate focus position, the selection of which hasbeen accepted.

The candidate displaying portion (part) 913 displays an image of theinspection object imaged upon adjusting the focus position to the onecandidate focus position, the selection of which has been accepted. Thecandidate displaying portion (part) 913 preferably displays candidatefocus position information concerning the plurality of extractedcandidate focus positions in a superimposed manner on the displayedimage. This is because it is possible to visually check whether or not adesired characteristic portion of the inspection object is clearlydisplayed at which candidate focus position, whereby a focus positioncorresponding to the desired characteristic portion can be selected.

An image processing unit 903 executes image processing on image datasubjected to focus adjustment. This corresponds to pre-processing onimage data captured by imaging, performed by the FPGA 131 of the mainsubstrate 13, such as brightness adjustment processing or white noiseadjustment processing.

In the present embodiment, inspection condition data in the case ofperforming focus adjustment is set and stored from the “setting mode”screen shown in FIG. 7B. The “SETTING NAVIGATION” button 75 is selectedfrom the “setting mode” screen shown in FIG. 7B, and the screen shown inFIG. 8A is displayed. The “Automatic focus adjustment” button 83 of FIG.8A is selected to perform focus adjustment.

FIGS. 16A to 16C are exemplary views of an automatic focus adjustmentscreen in the display device 2 of the image processing sensor accordingto the embodiment of the present invention. In FIG. 16A, a currentlyimaged image of the inspection object is displayed in a main displayregion 160. When an “Automatic focus adjustment” button 162 is selectedin this state, automatic focus adjustment is executed.

Specifically, upon selection of the “Automatic focus adjustment” button162, the DSP 132 transmits a rotation control signal to the motor driver181, to rotate the motor 141. The lens 12 is moved from a positionhaving a shortest distance from (closest to) the inspection object to aposition farthest from the inspection object, and the inspection objectis imaged for a plurality of times during movement. A plurality ofcandidate focus positions are extracted based on the plurality of piecesof imaged image data, such as the focus degree.

The lens 12 is moved to a candidate focus position which is the closestto the camera module 14, namely, having the shortest focus distance,among the plurality of extracted candidate focus positions, to adjustthe focus position to the candidate focus position and complete theautomatic focus adjustment. FIG. 16B is an exemplary view of anadjustment completion screen at the time of completion of the automaticfocus adjustment.

As for the image displayed in the main display region 160, both rightand left images of the displayed image of the inspection object areblurred in FIG. 16A, whereas the image on the right side is clearlydisplayed in FIG. 16B. This is because the lens 12 has been moved to thecandidate focus position having the short focus distance.

On the adjustment completion screen, candidate focus positions 164, 165,and a current focus position 163 are displayed. In the example of FIG.16B, the current focus position 163 agrees with the candidate focusposition 165. On the adjustment completion screen of FIG. 16B, one barwith its left end being “Far” and its right end being “Near” isdisplayed on the lower side of the screen, and buttons indicating thecandidate focus positions 164, 165 are arranged on the displayed bar inthe vicinity of the respectively corresponding candidate focuspositions. This allows the user to grasp at first glance how long thefocus distances of the candidate focus positions 164, 165 are.

Further, an inverted triangular figure indicating the current focusposition 163 is also arranged on the displayed bar in the vicinity ofthe corresponding candidate focus position (arranged at a positionpointing the candidate focus position 165 in FIG. 16B and at a positionpointing the candidate focus position 164 in FIG. 16C). This allows theuser to grasp at first glance how long the focus distance of thecurrently displayed image is.

In addition, for showing the current focus position 163, instead of theinverted triangular figure as shown in FIG. 16C, for example, a buttonindicating the current focus position 163 between a button indicatingthe candidate focus position 164 and a button indicating the candidatefocus position 165 may be illuminated. That is, in order to allow theuser to select a position among the plurality of extracted candidatefocus positions on the touch panel 21 of the display device 2, candidatefocus position information concerning the plurality of extractedcandidate focus positions (preferably, the plurality of candidate focuspositions and the current focus position) is displayed.

The user can perform an operation to change the current focus position163 to another candidate focus position 164, so as to move the lens 12to another candidate focus position. For example, by touching anothercandidate focus position 164 on display with a finger 166, the currentfocus position 163 is changed to another candidate focus position 164,and the image displayed in the main display region 160 is also changed.

FIG. 16C is an exemplary view of the adjustment completion screen in thecase of changing the position to another candidate focus position 164.In FIG. 16C, only the image on the left side is clearly displayed. Thisis because the lens 12 has been moved to another candidate focusposition 164 having a different focus distance.

Therefore, a plurality of candidate focus positions can be extracted byperforming the automatic focus adjustment once, and the user can selecta desired candidate focus position while viewing a displayed image.Further, the user is not required to specify an adjustment range, avirtual center point and the like, and can easily adjust the focusposition to the desired candidate focus position.

FIG. 17 is a flowchart showing a procedure for automatic focusadjustment processing of the FPGA 131 and the DSP 132 of the mainsubstrate 13 in the imaging device 1 of the image processing sensoraccording to the embodiment of the present invention. The FPGA 131accepts a start instruction for automatic focus adjustment (step S1701),and transmits to the motor driver 181 a movement instruction to move thelens 12 to an imaged position corresponding to an adjustment regionwhich is the closest to the camera module 14, namely, having theshortest focus distance (step S1702). The motor driver 181 rotates themotor 141 in accordance with the movement instruction, and moves thelens 12 to a position which is the closest to the camera module 14,namely, to a position farthest from the inspection object.

The FPGA 131 transmits an imaging instruction for the inspection objectto the CMOS substrate 142 of the camera module 14 of the imaging device1 (step S1703), and acquires an image of the inspection object which wasimaged by the CMOS substrate 142 having received the imaging instruction(step S1704).

The DSP 132 calculates the focus degree of image data of the capturedimage (step S1705). Herein, the focus degree is calculated using thefocus degree calculating method described in FIGS. 10 and 11.

The DSP 132 determines whether or not the focus degrees with regard toall of adjustment regions of the captured image data have beencalculated (step S1706).

When the DSP 132 determines that an adjustment region whose focus degreehas not been calculated is present (step S1706: NO), the DSP 132transmits to the motor driver 181 a movement instruction to an imagedposition corresponding to the next adjustment region (step S1707). Themotor driver 181 rotates the motor 141 in accordance with the movementinstruction, and moves the lens 12 to a next imaged position. The DSP132 returns the process to step S703, and repeats the above process.

When the DSP 132 determines that the focus degrees with regard to all ofadjustment regions have been calculated (step S1706: YES), the DSP 132extracts candidate focus positions (step S1708). Herein, the candidatefocus position extracting method described in FIGS. 12 and 13 is used.The DSP 132 transmits a display instruction for candidate focus positioninformation concerning the plurality of extracted candidate focuspositions to the touch panel 21 of the display device 2 (step S1709).

As the candidate focus position information, for example, it is notlimited to display of the candidate focus positions 164, 165 shown inFIG. 16. FIGS. 18A to 18C are exemplary views of a candidate-focusposition selection accepting screen of the display device 2 in the caseof displaying candidate focus position information by selection of acandidate button on the touch panel 21.

As shown in FIG. 18A, a thumbnail image or the like of image datacorresponding to the candidate focus position is previously displayed inan image display unit 180, and by accepting selection of “CANDIDATE”button 182 or 183, an image, the selection of which has been accepted,is displayed. For example, when the selection of the “CANDIDATE” button182 is accepted, an image shown in FIG. 18B is displayed. It can be seenthat a pattern on the left side of the image is in focus.

On the other hand, when the selection of the “CANDIDATE” button 183 isaccepted, an image shown in FIG. 18C is displayed. It can be seen that apattern on the right side of the image is in focus. Information of aplurality of candidate focus positions is simultaneously displayed inthis manner, and an instruction of the candidate focus positioninformation is accepted, thereby allowing the selection of the candidatefocus position to be accepted.

Further, even when the candidate focus position information is notdisplayed, the selection of the candidate focus position can be accepteddirectly on the touch panel 21. FIGS. 19A to 19C are exemplary views ofthe candidate-focus position selection accepting screen of the displaydevice 2 in the case of accepting the selection of the candidate focusposition by specification of a target focus position on the touch panel21.

As shown in FIG. 19A, an image of the inspection object is previouslydisplayed in an image display unit 191, and by accepting selection of aposition pointed by a finger 192 or 193, the lens 12 is moved to thecandidate focus position which is stored in association with a pixel ofthe position, the selection of which has been accepted, to display animaged image. For example, when the selection of the position pointed bythe finger 192 is accepted, an image shown in FIG. 19B is displayed. Itis found that a pattern on the left side of the image is in focus.

On the other hand, when the selection of the position pointed by thefinger 193 is accepted, an image shown in FIG. 19C is displayed. It isfound that a pattern on the right side of the image is in focus. In thismanner, it is possible to accept selection of the candidate focusposition without directly displaying information of a plurality ofcandidate focus positions on the screen.

Further, candidate focus position information may be made sequentiallyselectable. FIGS. 20A and 20B are exemplary views of a focused-stateselection accepting screen of the display device 2 where the focusedstate is made sequentially selectable.

As shown in FIG. 20A, an image of the inspection object is previouslydisplayed in an image display unit 201, and by accepting selection of a“NEXT CANDIDATE” button 202 or a “PREVIOUS CANDIDATE” button 204, thelens 12 is sequentially moved to the stored candidate focus position, todisplay an imaged image. For example, when the selection of the “NEXTCANDIDATE” button 202 is accepted, an image shown in FIG. 20B isdisplayed. The displayed image may be checked, and in the case ofsetting the image as the focus position, selection of a “CONFIRM” button203 may be accepted.

Further, the displayed image may be stored by setting a small region foreach of the same candidate focus positions, to accept selection in unitsof the small region. FIGS. 21A to 21D are exemplary views of thecandidate-focus position selection accepting screen of the displaydevice 2 in the case of setting a small region for each candidate focusposition.

As shown in FIG. 21A, an image of the inspection object is previouslydisplayed in an image display unit 201, and small regions 211, 212, 213are sectioned for respective candidate focus positions. Each of thesmall regions may be displayed in a color-coded manner on a screendisplay.

In this state, selection of each of the small regions 211, 212, 213 isaccepted. For example, when selection of the small region 211 isaccepted, as shown in FIG. 21B, a screen is displayed including an imagein which the small region 211, the selection of which has been accepted,is in the focused state. Similarly, when selection of the small region212 is accepted, as shown in FIG. 21C, a screen is displayed includingan image in which the small region 212, the selection of which has beenaccepted, is in the focused state. When selection of the small region213 is accepted, as shown in FIG. 21D, a screen is displayed includingan image in which the small region 213, the selection of which has beenaccepted, is in the focused state.

Needless to say, the candidate focus position may be extracted based ona high-frequency extracted image created by extracting a high-frequencycomponent of an image obtained by imaging the inspection object. FIG. 22is a flowchart showing a procedure for automatic focus adjustmentprocessing based on a high-frequency extracted image, performed by theDSP 132 of the main substrate 13 in the imaging device 1 of the imageprocessing sensor according to the embodiment of the present invention.The DSP 132 accepts a start instruction for automatic focus adjustment(step S2201), and sorts stored image data of the inspection object inthe order of lens positions (step S2202). The DSP 132 reads out imagedata of the inspection object at a first lens position (step S2203).

The DSP 132 calculates the focus degree of the read-out image data (stepS2204).

The DSP 132 determines whether or not the focus degrees with regard toall of stored image data have been calculated (step S2205).

When the DSP 132 determines that image data whose focus degree has notbeen calculated is present (step S2205: NO), the DSP 132 reads out nextimage data (step S2206) and returns the process to step S2204, to repeatthe above process.

When the DSP 132 determines that the focus degrees with regard to all ofthe image data have been calculated (step S2205: YES), the DSP 132extracts the candidate focus position (step S2207). Herein, thecandidate focus position extracting method described in FIGS. 14 and 15is used. The DSP 132 transmits a display instruction to displaycandidate focus position information concerning the plurality ofextracted candidate focus positions to the touch panel 21 of the displaydevice 2 (step S2208).

As described above, according to the present embodiment, a plurality ofcandidate focus positions are extracted and focus adjustment can beperformed while checking on a screen as to which part of the imagedregion is in the focused state. Accordingly, it is possible for the userto easily select a focus position corresponding to an image in which adesired characteristic portion of the inspection object is clear.

That is, in the imaging device 1 according to the present embodiment,the inspection object is imaged by the camera module 14 while a focusposition with respect to the inspection object is changed by the focusadjustment mechanism made up of the motor driver 181 and the motor 141,to acquire a plurality of pieces of different image data. Based on thecaptured plurality of pieces of image data, a plurality of positionswhere part of the imaged region comes into the focused state areextracted as candidate focus positions, and candidate focus positioninformation concerning the plurality of extracted candidate focuspositions is displayed in the display device 2. This allows the user toeasily select a focus position corresponding to an image in which adesired characteristic portion of the inspection object is clear.

It is to be noted that the timing for imaging the inspection object andacquiring a plurality of pieces of different image data and the timingfor extracting a plurality of candidate focus positions are notparticularly limited. For example, all of the plurality of pieces ofdifferent image data may be captured first, and a plurality of candidatefocus positions may be thereafter extracted using all of the capturedimage data. Alternatively, it may be previously decided, for example, toextract three candidate focus positions, and acquirement of image dataand extraction of candidate focus positions may be alternatelyperformed, and then completed at the time of extraction of threecandidate focus positions.

Describing the latter case more specifically, image data is captured,and when one candidate focus position is extracted, the extractedcandidate focus position is stored, and when a candidate focus positionis to be extracted next based on the captured image data, it isdetermined whether or not it is a position different from the alreadystored candidate focus position. When the position is determined to be adifferent position, it is extracted as a new candidate focus position.

On the other hand, when the position is determined to be the sameposition, it is not extracted as a new candidate focus position (it isregarded as the same candidate focus position as the already extractedand stored candidate focus position). Such processing may be repeatedand the acquirement of image data may be completed after extractingthree candidate focus positions.

The present invention is not limited to the above embodiment, and avariety of changes, modifications, and the like can be made as long asit is within the scope of the gist of the present invention. Forexample, the imaging device 1 and the display device 2 are not limitedto the form of being directly connected through the connection cable 3,and needless to say, the imaging device 1 and the display device 2 maybe connected via a network such as LAN, WAN, or the like. Further,although the imaging device 1 and the display device 2 are separatedbodies in the present embodiment, the imaging device 1 and the displaydevice 2 may be combined to form an image processing apparatus.

What is claimed is:
 1. An image processing apparatus, comprising: animaging unit for imaging a region including an inspection object; adisplay unit for displaying an image imaged by the imaging unit; a focusadjusting unit having a focus adjustment mechanism for adjusting a focusposition with respect to the inspection object; and an image processingunit for executing image processing on image data captured by imagingwith the imaging unit, wherein the focus adjusting unit includes acandidate focus position extracting portion for extracting as candidatefocus positions a plurality of positions where part of the imaged regioncomes into a focused state based on a plurality of pieces of differentimage data captured by imaging the inspection object by the imagingunit, while changing the focus position with respect to the inspectionobject by the focus adjustment mechanism, and a candidate displayingportion for displaying candidate focus position information concerningthe plurality of extracted candidate focus positions in the displayunit.
 2. The image processing apparatus according to claim 1, whereinthe candidate displaying portion displays in the display unit thecandidate focus position information concerning the plurality ofextracted candidate focus positions so as to be selectable by the user,the focus adjusting unit is provided with a selection accepting portionfor accepting selection of one candidate focus position among theplurality of candidate focus positions displayed in the display unit,and when the selection of one candidate focus position is accepted bythe selection accepting portion, the focus adjusting unit displays inthe display unit an image in which the focus position is adjusted to theat least one candidate focus position, the selection of which has beenaccepted.
 3. The image processing apparatus according to claim 1,wherein the focus adjusting unit is provided with a selection acceptingportion for accepting selection of one candidate focus position amongthe plurality of extracted candidate focus positions, and when theselection of one candidate focus position is accepted by the selectionaccepting portion, the focus adjusting unit adjusts the focus positionto the one candidate focus position, the selection of which has beenaccepted.
 4. The image processing apparatus according to claim 1,wherein the focus adjusting unit is provided with a selection acceptingportion for accepting selection of one candidate focus position, andwhen the selection of one candidate focus position is accepted by theselection accepting portion, the candidate displaying portion displaysin the display unit an image imaged upon adjusting the focus position tothe one candidate focus position, the selection of which has beenaccepted.
 5. The image processing apparatus according to claim 1,wherein the candidate displaying portion displays in the display unit acandidate focus position having a shortest distance from the imagingunit among the plurality of candidate focus positions.
 6. The imageprocessing apparatus according to claim 1, wherein the focus adjustingunit is provided with a focus degree calculation portion for calculatinga focus degree for indicating a degree of a focused state with regard toeach of a plurality of previously set small regions, and a focusposition acquiring portion for acquiring a focus position with a maximumfocus degree among the focused degrees each calculated for each of thesmall regions, and the candidate focus position extracting portionextracts the candidate focus positions based on the acquired focusposition for each of the small regions.
 7. A focus adjusting methodexecutable by an image processing apparatus, the apparatus including: animaging unit for imaging a region including an inspection object; adisplay unit for displaying an image imaged by the imaging unit; a focusadjusting unit having a focus adjustment mechanism for adjusting a focusposition with respect to the inspection object; and an image processingunit for executing image processing on image data captured by imagingwith the imaging unit, wherein the focus adjusting unit extracts ascandidate focus positions a plurality of positions where part of theimaged region comes into a focused state based on a plurality of piecesof different image data captured by imaging the inspection object by theimaging unit, while changing the focus position with respect to theinspection object by the focus adjustment mechanism, and displayscandidate focus position information concerning the plurality ofextracted candidate focus positions in the display unit.
 8. The focusadjusting method according to claim 7, wherein the candidate focusposition information concerning the plurality of extracted candidatefocus positions are displayed in the display unit so as to be selectableby the user in the display unit, the focus adjusting unit acceptsselection of one candidate focus position among the plurality ofcandidate focus positions displayed in the display unit, and when theselection of one candidate focus position is accepted, the focusadjusting unit displays in the display unit an image in which the focusposition is adjusted to at least one candidate focus position, theselection of which has been accepted.
 9. The focus adjusting methodaccording to claim 7, wherein the focus adjusting unit accepts selectionof one candidate focus position among the plurality of extractedcandidate focus positions, and when the selection of one candidate focusposition is accepted, the focus adjusting unit adjusts the focusposition to the one candidate focus position, the selection of which hasbeen accepted.
 10. The focus adjusting method according to claim 7,wherein the focus adjusting unit accepts selection of one candidatefocus position, and when the selection of one candidate focus positionis accepted, the focus adjusting unit displays in the display unit animage imaged upon adjusting the focus position to the one candidatefocus position, the selection of which has been accepted.
 11. The focusadjusting method according to claim 7, wherein the candidate displayingportion displays in the display unit a candidate focus position having ashortest distance from the imaging unit among the plurality of candidatefocus positions.
 12. The focus adjusting method according to claim 7,wherein the focus adjusting unit calculates the focus degree forindicating the degree of a focused state with regard to each of aplurality of previously set small regions, acquires a focus positionwith a maximum focus degree among the focused degrees each calculatedfor each of the small regions, and extracts the candidate focuspositions based on the acquired focus position for each of the smallregions.